Striving Readers Cohort I] 
Evaluation Report: 
Kentucky 


SR Cohort II Evaluation Report: Kentucky 


Acknowledgments 


The authors gratefully acknowledge the contributions of the many individuals who 
facilitated this evaluation. This evaluation could not have been successful without the support 
of Cindy Parker, Robin Hebert, Mary Rudd and Monica Osborne from the Kentucky Department 
of Education (KDE) and administrators and staff in the nine Striving Readers schools. We want 
to thank the Striving Readers interventionists who work diligently with students and teachers 
each day and to whom we turn regularly for information to further this work. Furthermore, we 
gratefully acknowledge the many students who participated in this study. 


Additional members of the evaluation team who deserve credit for their contributions 
to this report include the following individuals to whom we are immensely grateful: 


Data Analyst 
Deepshikha Sigdel 


Research Assistants 
Laura Dudney 
Amanda Goodwin 
Jessica Grewe 
Melissa Murphy 
Jennie Pahl 


Staff Support 
Elisha Comer 
Judy Johnson 


Finally, we extend our gratitude to the staff of the Collaborative Center for Literacy 
Development (CCLD) for their ongoing support of this work, Abt Associates Inc. for the technical 
assistance they provided during the study, and the U.S. Department of Education for funding 
this project. 


Cantrell, Carter & Rintamaa, 2012 


SR Cohort II Evaluation Report: Kentucky 


Table of Contents 


Executive Summary of Findings: Implementation and IMpact.............csssssscecsrceseessseseesees 
KCLM Targeted INtervVention.........cccccccssesssesssecsecsecsecesssscessesssssesesessecoesoeseseesaesaeseneceeseeseaseaaeees 
IMA PIEMVEN TAU! OF secs secescoesssscecca ate tics cegeeesveetetsvecen sti Geaees saltecd haustiieasctete Gevtigttesesseevieeactataes 

WIV ACCS sae vss csane hecees ca desecnes Sach es obaSh caalceastivadl Gelaas ctaaceadstiea sack iaieuatadavssoecsasteteans chvascaasteaesbadas 


Introduction and Study Background..........sssssccsccsssssssessscsscsscesscssessesceseesscsseesescssassseesscseessesens 
Description of the Intervention MOdel.........cccccccccscsscessessessessecesececsesessessesseseescaeeseseessesesees 
Framework of the Supplemental Literacy Intervention ClaSS.........c cscs cseeeeeeeeeeeeeeeeeees 
Components Of KCLM hu... .ccecesssssessessseseecsecsesesssscessessescaessecsesseseseesssseessaesaecoeseesesaeseuseeeseeceeses 
WeaP Sted SOUS i 8 cscs vast snctts dmc ts tenstadecen nets Gunes cesta en doitac baresbnacunsleathe veinmesatin tecedecmedacesaniens 
Selection. Process TOF IMtEFVENTIONMISES :.ccscssscsciusescaayadiaidiaarduecisieed desesdedeeredendnweaetienddad 
Desired Characteristics of the Intervention ClaSSrOOM.......ccssssseceseeceecesceeceeceeceeceeeeecaeees 
Logic Model for Kentucky Cognitive Literacy Model Intervention. ...........:cscsseeeeeeeeees 
Planned Tralmtrig: MaOGe YN ssass 65 cs cidec sess scensccaccawas cuncccwiecaacey ae sesexaduasah esraseaieue eensecaestiauaeiesuniiasia 
Planned Classroom Instructional Modelscccsicccscccsscscstsscsssecesssssaccasteecncaseasenss ssessenss teseneeaweneeoeds 
Planned Experiences for Control Student...........:ccscscceccsecsecseceseeceesesceseeseeeseseseesaeseessesaeeaees 
Unit Planning Template...........c.ccccccsecsecescsscessesssssesscecoeceeseseessesseseeeaeeceeseesesseesesesseescaesaeseessees 
Key Evaluation Design Features.........ccccccsssssessecsscsscsssssssssessessesees coecseceeseeseseessescaeeseseeseseseees 


Evaluation of the Implementation.............cssssccsscssssscsscscsessecsscesscssceeecessessesseessscessesseesassseeeeees 
Summary of the Design of the Implementation Study...........ccceesssssecsecseceeseesessesseseseeeaes 
Implementation Data Collection and AnallySis.........c.ccccscsseseseesecsscssssssescesseseecsecseseeseeeessees 

SUMMUIMIOL TAMING sis sscsessececveiecscvstess ivtekiaeededeeaasarssssvaed cave veckeaslevadesiacseuasass dedanssd0evdesastevieseaees 
COaGIiIM GIILOMI Mie seescececee: ces cs sa sates sas cece tant Soa ces etna tai ose ae castes a a ete 
RFeaS SOO TaW UNGER UC TON R as leaeics cadets test denaenctonc cunecadsdadedesdbansteduemavtounieidetenabeasepet ences Beseeeuaces 


KCLM Implementation RESUItS ..iscscncesicccscacsacossssessccssecncccnsesssarsncecnssscnsenessssctearussedssioduncavonsasrens 
Characteristics of Intervemntlonists sccsccosscacsaccssisioxestetax sannpeats lense Gacasanseudelbuldavsetauasacyetaudedewuaseus! 
Implementation of Professional Development MOdel..........csecesceeccssceececeesseceeeeeeeseesees 

INTEPVENTIONISES TPAINING...........cccsscccessecsrsccssecesssecescsssessceceeseecenseessesecseeseserscessescessuecenanees 
Professional deVeElOPMeNt INPUtS...........cccsecsecsscsscescescessesseseeeeeceeceesesseesesseseeecaesaeseeseeees 
Professional development for AAMINIStFATOSS............ccssecesseeceeceeceeceeceecaecaeceecaecaeeeeeasens 
Implementation of Classroom MOdel sists sccscsaiscidsesessticcassattinesieseceasinsccanstescensesecaonsinsiesoeatns 
Class size, intensity, ANG CUIatiON...........ccccesesecsessecsecsecessessesseseeesescaecaessecsesesaesuscaseaees 
Classroom implementation reSults..........ccccccsecsscsscssssssessesseseesescsecseceesseseseeseeseeeseeeeeens 
Experiences for control students during intervention PeriOd..........ccccecceessecseeeeseeees 
Additional Treading PROLCA IS vse ccicsevecess suscscasaccn ce cassvsnoesdeeinisevsnea sapetevectistacsnetiesctahememetancsel 
Implications for impact ANALYSIS...........cccsseseeeseececssceecessessessesseesesceesseceeeessessessescaesaeseeeeees 


Evaluation Of Impact tveaiccieccccsictincidicotccnnaienecnestivlennnnnasiecnionas 
SEW IDS SI 81a. oz ccerccsdtehascuuecasnovctiehdvcuatechancuassastacaanedrcunte spuceseeaignea autos a nesectiamdccasAakonte eens 


Cantrell, Carter & Rintamaa, 2012 


SR Cohort II Evaluation Report: Kentucky 


Sampling Sel@ction PrOCeS........cccccsccecescescessessessesseecseseseessessesseeaeeceecseseeesessesaesaeecaeesesaesees 
SAM OS1 ZG osie53 csiesv cede aeaes duces ee ssesesessevacussvssdsay sevens Soavaceezesuetaseegidssisusaseseesssavannsuiveed easels 
Consort Diagram Reading AchieVeMent..........ccccecescescseceecseceeceecaeseeceseesceseeeeeaesaeseeeaes 
Consort Diagram Writing ACHieVeEMEeNL...........ccccccecessessessesssesesceecseceesessessessesceesaecneens 
Consort Diagram Student Efficacy aNd Motivation..........cccccecseceeceeceececeessecaeeeeeeseeeees 
Metacognitive Awareness of Reading Strategies Inventory (MARSI).............c:cce8 
AGOlESCaNE IMIGtIVATION: SUIVOY .cesiccsnececacsess caaivendantandesdczetemsencaeesiaceedeudceadeedaterscatenteresta 
Summary of Analytle Approac nice sicteccietissntertesscccsacu tarsi ontseniatvacerntierels ence eines 
bevel k LIVIS Sie nin We Vel scars sonnasisrcs aeaetevencepdasencandacietedteltehente eaeeeaseaceaetavia 
Level? TALI SCO WV iss ceccecauess ctecaesttececeepdeexencestecurteticcndaenendsniaubieantendemenioes 
Descriptionvorthe First Year Samples scsiicsesoecarbesssicstdcauscteideea sivas Aceclanasdestetecesseavemnnesaant 
PVUACES OTN SUI INS sche cacca scence lato ieaceesa ccecca aan aca-Ginedeoest iocacestenceas sacanrndecepuenemeceeareecsestei seem aaes 
DISCUSSION ANNE COMEIUSIONS 5 iciisccsatiecsiicscessucscnasceranctncce cdi essazsnveieascabdacescietibdaxslancadhabideaxaarnavaanneeix 
PGT N NCES ccc catemssceceneteissewsttn ceeaiee Guu ssneck ca unsantciesaasnecnscuccasutyeccsanscesavasensucbsrausttanaanteverenasaeeiies 


APPENDICES 
As Summary of Analytic Procequtes sissccssec saccerccnaiciceacosevecsatshesiubivisencuneeuine avatsemcatennvaanedt 
BS UN WS UNF Soca daycare edeaaceas dace uicantnsg svaabegedee deo te nce nada eae en ed ante nae i aaa acme 
SULTANS Sy cassette cence nee wine dh ce cca gts ence gp candace edad eet eeeteee de cutee 
Teacher ObServation: Protocol sa scccasicdcetinslausd dtatauaisnariaaacesamnsaaded 


EXHIBITS 
1: Consort Diagram Reading AChieVeMe nt............cccsscsssesessessessesesecssseeceeceeseeceeeecseceeseesasaees 
2: Consort Diagram Writing ACHI@VEMENL............cccccecsecescescessesssssesccesseceecesessesseseeeeneceseaes 
3: Consort Diagram Student Efficacy and Motivation. ..........ccccsecssscceeeeseesessesestesesseeseeees 


FIGURES 
TF COMPOMENIHS OF KOLM se sesiescvsecsacessevexaucivevessvanzesstucesadsdsevcs sei tsesaksevbesusducvo seeieiaeuevavessiayeivessiss 
2: KCLM Intervention Logic MOdel ..........ccccccesssecsecsesssssssessessesssessecsscsecsecsecesasecescesescesaeeceseseseseesaes 
3: Unit Planning Termplate...............:cccccccsccsscsscsssesssssssceeceecsecsscessessessesenessessessecesaesensaneatessesses 


TABLES 
1.1 Framework of the Supplemental Literacy Intervention Class............ccce cscs seeeeeeeees 
1.2 Plame PD ACtiV tie Sos ccccsissss cord csteccsscassnstesscoteatsesecrdeaceaonaranteaie¥edtashiene eared 
2.1 Content and delivery of professional development training (KCLM).............c.ceeeee 
2.2 Most Widely Implemented Model COMPONe nts ...........csccsecsscsesessescessesseeseecseecseeaeeees 
2.3 Mean Scores by Teacher for Quality of KCLM COMPONENL...........:ccccccceccecceceeeeeeeeeees 
2.4 Intervention and Control Student Demographics (and Proportions).............c:ccce 
2.5 Impact of the Target Intervention on Student Reading Achievement, Writing 
Achievement, Strategy Use Self Efficacy, and Motivation............scsssecsseseeceeeeceeees 


Cantrell, Carter & Rintamaa, 2012 


SR Cohort II Evaluation Report: Kentucky 


Striving Readers 
Cohort Il: Kentucky 


Executive Summary of Findings: Implementation and Impact 


This Striving Readers evaluation examined the impact of a targeted intervention for 
struggling adolescent readers in participating schools. This study was conducted in nine high 
schools in nine school districts serving large percentages of at-risk students in Kentucky. The 
targeted intervention for struggling readers was the Kentucky Cognitive Literacy Model (KCLM) 
developed by the Kentucky Department of Education. 


The effectiveness of the KCLM intervention was determined through a randomized 
control field trial utilizing a treatment and control group design. The KCLM was a supplement to 
the regular curriculum wherein students in the targeted intervention participated in a reading 
class in place of an elective as part of their regular school day. The control-group condition was 
“business as usual,” wherein students in the control group take a regular elective such as band, 
theater arts, civics, or physical education. This study examined the implementation of KCLM 
and its impact on struggling ninth-grade students’ reading and writing achievement, self- 
efficacy with reading strategies, and motivation for reading. 


In this Striving Readers project, each school employed an intervention teacher who was 
responsible for teaching the targeted intervention to struggling readers. 


The impact research questions that motivated the study design and analysis plan are: 


e What is the impact of Kentucky’s Cognitive Literacy Model (KCLM) on the reading 
achievement of low-achieving readers? 

e What is the impact of KCLM on the writing achievement of low achieving readers? 

e What is the impact of KCLM on the perceived reading strategy use of low achieving 
readers? 

e What is the impact of KCLM on the motivation and engagement of low achieving 
readers? 


In addition, the following implementation questions will be answered: 


e What is the state-level implementation of the professional development training and 
support for intervention teachers in the project? 

e Which components of KCLM were implemented most frequently by teachers in 
classrooms? 

e What was the quality of KCLM implementation in classrooms? 
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KCLM Targeted Intervention 


This study examined both the implementation and impact of the KCLM over the course 
of the first year of the Striving Readers project. 


Implementation. During the year of the project, KCLM teachers participated in training 
and on-site support, and training was provided for school administrators. During the year, 
KCLM teachers were provided 11 days professional development training and up to 66 hours 
support from visits by KDE literacy staff. School administrators were provided one day of 
training and at least two support meetings regarding the intervention for the year. Overall, 
participation in the professional development the KCLM teachers and administrators was high, 
with 100% of teachers and administrators participating fully in the training either through the 
formal training dates or through makeup training. Classroom implementation fidelity was 
measured through classroom observations. Observations indicated that teachers implemented 
some components of the KCLM model more readily than other components. The intervention 
was implemented with adequate quality in 3 of 9 classrooms. 


Impacts. This study used hierarchical linear modeling (HLM) analyses to measure the 
impacts of the KCLM on ninth-grade (high school) students’ reading achievement, writing 
achievement, perceived strategy use, and motivation. In this Striving Readers study, there were 
no impacts on students’ reading achievement as measured by the Group Reading Assessment 
and Diagnostic Evaluation and no impacts on writing achievement as measured by the Kentucky 
State Writing Assessment. Student survey results indicated significant effects of the 
intervention on participating students’ self-efficacy for strategy use and on students’ reading 
motivation. 


Introduction and Study Background 


Description of the Intervention Model 


The targeted intervention for the Kentucky project was the KCLM, developed by the 
Kentucky Department of Education (KDE). The purpose of the KCLM was to assist students who 
were significantly behind grade level in reading by providing them with the supports to be 
successful in learning across the curriculum. Table 1.1 illustrates the strands of the 
intervention. 
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Table 1.1 


Framework of the Supplemental Literacy Intervention Class 


Motivation and 
Engagement 


Strategic 
Processing 


Instructional 
Strategies for 
Content Learning 


Communication 
Skills 


Thematic instruction with project/inquiry-based learning produces 
students who are more fully engaged and motivated to learn. 

Literacy is a social accomplishment (Bloome, 1986; Dyson, 1992). 
Strategy—a deliberate cognitive process of selecting, enacting, 
monitoring and regulating behavior. Includes comprehension strategy 
instruction and foundational reading skills. 

Skill—a mental activity that can be applied to specific learning situations. 
Metacognition—key to strategic processing because it enables students 
to monitor progress toward achieving their goals (Flavell, 1979). 

Robert Marzano’s characteristics of effective vocabulary instruction 
Marzano, et al. strategies for learning such as cues, questions, and 
advance organizers; non-linguistic representations; identifying similarities 
and differences; summarizing and notetaking 

Includes reading, writing and discussion outcomes that address a variety 
of approaches to essential questions and texts. 

Writing helps readers clarify meaning and provides opportunities for 
authentic engagement and communication. 

Exchanging ideas, especially through extended discussion of meaning 
and interpretation of text, is essential to a learning community. 


It is important to note that while these are categorized into specific strands, they also are 
embedded across strands to integrate a total literacy experience for students. The four 
components of the KCLM framework were tied together by content-related themes such as 
success, the environment, and problem solving. 
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Figure 1 shows the major course components and specific dimensions of each component. 


Supplemental Intervention Model Components 


Motivation & 
Engagement 


Strategic Processes 


Instructional 
Strategies 


Communication 


Theme-based Instruction 


Connections between 
learning activities and 
real world issues 


Varied instructional 
format 


Student prior 
knowledge, interest 
and background used 
in determining 
content 


Technology is used to 
facilitate learning 


Goal setting and 
private feedback 


Focus on problem- 
solving processes 


Autonomy for 
learning and 
meaningful choices 


Explicit 
comprehension 
strategy instruction 
(modeling, 
explanation, practice, 
and reflection) 


Explicit instruction in 
foundational reading 
skills 


Instructional tools for 
comprehension and 
vocabulary learning. 


Vocabulary 
processing through 
visual, auditory, 
physical and/or 
emotional 
experiences; 
opportunities to use 
their own words or 
non-linguistic 
representations to 
define new words; 
teacher explanations 
and examples of new, 
key terms 


A focus on 
summarizing and 
identifying similarities 
and differences 


Higher level 
questioning 


Pre-, during-, and 
post-text based 
discussion strategies 


Writing to learn 
activities 


Explicit instruction in 
writing strategies 


Explicit instruction in 
foundational writing 
skills 


Figure 1: Components of KCLM. 
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In the Kentucky project, ninth-grade students who scored two grade levels or more below 
grade level in reading received a minimum of 225 minutes per week and a maximum of 375 
minutes per week of supplemental reading instruction in a targeted intervention class taught by 
an intervention teacher. Students were placed in this course in addition to their regular 
reading/language arts classes for an entire school year. 


Over the course of the project, the professional development model for the targeted 
intervention included summer and follow-up trainings and on-site support from KDE literacy staff. 
To learn how to implement the targeted intervention, teachers participated in a summer 
workshop, which was led by KDE literacy staff. During the school year, the trainers led the 
teachers in follow-up workshops. Across the project, KCLM teachers received 11 days of workshop 
training in the targeted intervention in total. To support their ongoing learning and development, 
teachers also participated in site visits and regular distance support by KDE literacy staff. 


The professional development model included training and support for administrators, as 
well. School administrators attended a one day meeting in the summer to learn about the KCLM 
model and the ways in which the schools should support the intervention. Additionally, KDE 
literacy staff participated in on-site meetings with administrators up to six times during the school 
year. Topics at those meetings included grant requirements, evidence-based components of the 
targeted intervention, scheduling issues, observations of interventionists, literacy leadership, 
literacy planning, and meeting the needs of struggling adolescent learners. Administrators also 
received KCLM update newsletters from KDE four times during the course of the year. 


Targeted Students 


This project was designed to serve low achieving readers in the ninth grade. In the spring of 
their eighth-grade year, students in middle schools that feed into the nine participating high 
schools were given the middle school form of the spring Group Reading Assessment and 
Diagnostic Evaluation (GRADE). Students that scored an NCE of 40 or lower on the GRADE were 
defined as low achieving readers, and were placed in the eligibility pool. 


Students in all day resource classes were not eligible; all other students were eligible. The 
evaluation team directed the faculty at all feeder middles schools to identify students in all day 
resource classes, and instructed the schools not to give the spring GRADE to these students. In 
addition, evaluators directed the faculty at the nine participating high schools to identify students 
that were placed in all day resource classes after they enrolled in ninth-grade this fall and asked 
them to provide evaluators with the names of these students for removal from the study. Finally, 
middle schools sent home passive consent forms with all eighth-grade students who would be 
tested for participation in the study. The study was described, and parents were directed to 
contact the evaluators if they chose to not allow their student to participate in the study. No 
parent declined permission for their child’s participation in the study. 


Two thousand two hundred four students were listed on the school register sheets as 
enrolled at the feeder middle schools. Evaluators received an additional twenty-five student 
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GRADE sheets that were not on the enrollment lists. There were 637 students that met the study’s 
criterion on the spring GRADE for eligibility. 


Selection Process for Interventionists 


Intervention teachers were recruited and hired by individual schools. Advertisements for 
the intervention teacher position included the following criteria: experienced classroom teacher, 
respected by faculty and administration; familiarity with and/or interest in interdisciplinary and 
project-based learning; willingness to learn and apply new skills and knowledge; 
planning/reflecting skills; strong leadership ability; adaptability and problem solving skills; 
presentation skills; collaboration skills; ability to mediate between the school and community 
organizations; and personal communication skills. 


Desired Characteristics of the Intervention Classroom 


Classes were to be no larger than twenty ninth-grade students. The intervention class was 
to meet daily for at least 45 minutes for the entire year. 
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Logic Model for Kentuc 


Training and Support for 
Interventionists 


© 5-day Summer Training 

Academy. (Year 1) 
KCLM Framework 
Strategic Processing and 
Academic Literacy 
KCLM Unit and Lesson 
Planning Templates 
Project-Based Learning, 
Classroom Management 
and Motivation 
KCLM Unit Topics 
Comprehension and 
Vocabulary Strategies 
Communication Skills 
Progress Monitoring and 
Formative Assessment 
Data Analysis and 
Foundational Reading 
Instruction 

3-5 day Summer Training 

Academy. (years 2-4) 

Online Courses and Professional 

Learning Community. 

Continuing Support by KDE 

Staff. 

16+ hours On-site and 50+ hours in 

distance support (email, phone, and 

webinars) 

Classroom Libraries and 

Instructional Materials. 


Training and Support for 
Administrators 


1-day Summer Institute (years 1-4) 
Site Visits by Key KDE Personnel 
On-going Literacy Leadership 
support by KDE staff 


Figure 2: KCLM Intervention Logic Model 


Intervention Classroom Variables 
(coded by # to Student Outcomes) 


Collection, analysis and interpretation of data to inform 
instruction in foundational reading and communication skills 


Students set personal learning targets and self-monitor 
progress 


Daily vocabulary instruction around Robert Marzano's 
6- step vocabulary plan, CODE, and/or vocabulary 
games and strategies 

Fre quent instructional use of cornprehension strategies 
(Cues, Questions, Advance Organizers, Non-Linguistic 
Representation, Identifying Similarities and 
Differences, and Summarizing and Note-T aking) 

(1) 


Project-Based Units & Lesson Plans include student 
choice , high-interest topics and social collaboration 


(3) 


Student progress in reading and communication skills is 
monitored through periodic diagnostics, formative and 
summative assessments, and pre- and post-tests 

(4,5) 


Cognitive and academic literacy skills (reading/studying; 
reflecting; thinking; communicating) are integrated into daily 
activities and strate gic processing (meta-cognition, making 
connections to/accessing prior knowledge, inferring and 
predicting, asking questions, determining important ideas 
and summarizing, visualizing, synthesizing and retelling, 
monitoring and clarifying of text) is modeled and rehearsed 
(1,2,3,4,5) 


Frequent collaborative and independent engagement in 
writing and discussion 


(4) 


High-Interest, leveled texts and instructional materials 
readily available to intervention teachers and students 


(3.4) 


Class meets for 50+ minutes each day, all year, 15-20 
students per class 
(4,5) 


Cognitive Literacy Model Intervention 


Increase in independent 
comprehension strategy use 


*Increased problem-solving and 
collaboration skills; increased 
transference of skills/strategies/ 
knowledge from intervention class 
to content area learning 


Increased motivation to read/learn 
and increased self-efficacy among 
intervention students 


Increased achievement in reading, 
writing and communication skills 
among intervention students 


. *Fewer struggling readers in 


participating schools 


Rev. 4/10 
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Planned Training Model 


During the summer of 2010, the intervention teachers from each school were to receive five 
days of initial training on the core strands. This training was to be developed and provided by the 
Kentucky Department of Education literacy staff. School administrators were required to attend 
one day of summer training to learn about the goals of the intervention and the expectations for 
support of the interventionist and students. By the end of the initial training, the interventionists 
were to create an instructional plan for direct strategy instruction, and integration of strategies 
into standards-based units of study. Each participant was to leave understanding how to teach, 
use, and assess within the KCLM framework, and with a plan to begin immediate implementation 
as the 2010-2011 school year began. 


Throughout the year of implementation, the interventionists were to receive on-going 
training and support through face-to-face meetings and an online Kentucky Virtual Schools (KYVS) 
learning community, which was to include book studies, Webinars, a discussion forum, and 
repositories for sharing resources. KDE literacy staff planned site visits to the schools in the fall 
and again in the spring to provide on-site coaching and support for up to 50 additional hours. 


Administrators were to engage in professional development focusing on the guide from 
the National Association of Secondary School Principals, Creating a Culture of Literacy: A Guide for 
Middle and High School Principals as well as a KDE DVD resource, Literacy Leadership: Stories of 
Schoolwide Success. This shared leadership network was intended to help administrators and 
coaches develop a close working relationship critical in supporting instructional improvement 
efforts. 


Table 1.2 
Planned PD Activities 
PD hrs. Activities Date Attendees * 
40 Training on core strands of intervention; develop July 2010 | 
instructional plan and units of study 
8 Training on goals for intervention and on July2010 A 
providing support to interventionists 
50+ Ongoing PD: Online learning community, 2010 | 


including book studies, webinars, discussion 
forum, sharing network 
16 On site visits/coaching 2010 


*l=Interventionist; A=Administrator 
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Planned Classroom Instructional Model 


The Kentucky Department of Education created a Unit Planning Template (UPT) to assist 
intervention teachers in creating units of instruction for the intervention class. One component of 
the UPT listed the strategies and activities that comprise the KCLM. Teachers selected strategies to 
teach in each unit. Figure 3 shows the strategies and activities as listed on the UPT. Teachers were 
free to select the strategies they taught based on their assessment of students’ needs. As part of 
the intervention class, students assigned to the intervention will take the SAT-10 online (reading) 
and TOWL-4 (writing) diagnostic assessments twice per year. 


In addition, KCLM is designed to engage students in reading a variety of level-appropriate 
texts related to essential question(s) for each content-related unit, including nonfiction, 
informational and procedural documents, and narrative texts. 


Planned Experiences for Control Students 


Expectations for the control students were “business as usual,” wherein students in the 
control group took a regular elective such as band, theater arts, civics, or physical education. 
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Cues, Questions and Advance Organizers 
O Anticipation /Reaction Guide (PK, 0, m/c, my) 
O Questions and Answers (PK, 1/P,0,5, M/C, M) 
O Skimming (Pk, /P, ¥, S/R) 
Non-linguistic Representations 
O Pattern Organizers (s, wv 
O Mind Mapping (a, v,5/r, m/c 
O Pictograph (pk, v, uj 
Identifying Similarities and differences 
Comparison Matrix Chart (PK,s, i) 
Comparison Guide Map (PK,s, v,5/R, M/C, M) 
Graphic Organizer for Analogies and Metaphors 
dP, S,V, Ml) 
Student or Teach er-Generated Classification 
Graphic Organizer (dependent on graphic organizer) 
Summarizing and Note taking 
Cornell Notes (0,5, mc) 
Note Taking Using Both Sides of the Brain (Pk,s, v, 
M/C 
Summary Frame (Dependent on frame} 
Rule- Based Strategy (o,5, u/c 


‘The letters in parentheses indicate the strategic processing skills that 
each strategy supports. 


Journal/Learning Log 

Graphic Organizers 

Rough Drafts 

Checklists (Introducing, Progressing, 
Mastering, etc.) 

Review Games/ Activities 

Teacher Questioning 

Student Practice Activities/Ex ercises 


Quizzes/Tests 


Role-Playing 
Other 


OOoooooo 


Figure 3: Unit Planning Template 


Exit Slips/ Activities 

Student Partner Talk (Think-P air-Share; Turn & Talk) 
Preliminary Plans/Outlines/Prototypes 

Practice Presentation 


Teacher's Anecdotal Notes 


Strategic Processing 

PK Making connections to prior knowledge and 
developing schema 

I/P Inferring and predicting 

Q Asking questions 

S Determining important ideas and summarizing 
V Visualizing 

S/R Synth esizing andretelling 

M/C Monitoring/clarifying understanding of text 
M Metacognition 

Collaboration 

O Paired 

O Group (Reciprocal Teaching) (1/P,0,5, m/c 

Fluency 

O Oral 

O Written 


al et et at sO 


Progress Work Report : Individual 
Progress Work Report: Group 
Student Goal setting 
Journal/Learning Log 


Writing 


Rubric 


Rubric 


O Writingto Leam 
O Writingto Demonstrate Learning | O Group Discussion 
O Writing for Authentic Purposes 


Written Project, with 


Oral Presentation, with 


Word Storming (Px, 1/P) 

Closed /Open WordS ort (Pk, 0,5, ¥,5/R, M) 
Three-Way Tie (PK, I/P,0,¥, M/C, M) 

Connect the Words (pK, 1/P, 0, v,5/R, M/C, M) 
LitFig (PK, 1/P,0,¥, M/C, M) 

Four S quare Vocabulary Map/Frayer Model 
(PK, ¥, M/C) 

Word Questioning (PK, 1/P, 0,5, V,5/R, M/C, Mj 
Text Impressions (PK, 1/P, 0, Ml) 

Cin quain (PK,S,¥,5/R. M/C, Ml) 

Rate Your Knowledge (P/K, IP, a/ M/C, M) 
Vocabulary Tree (Pk, m/c, M1) 

Games 

O Password (pk, 1/P, v, S/R) 

O Memory (Pk, mc 

O Most important Word (pK, 1/P, 0,5, Ml) 


*The letters in parentheses indicate the strategic processing 
skills that each strategy supports. 


a a a 


Speaking 
O Paired Discussion 


O Presenting to an Audience 


Ol Peer Evaluation 
Os Self-Evaluation 
O Other 


On-Demand Writing 


Test 


Adapted from The Unit Organizer Rounne (Edge Enterprises, 1994); PSL Storter Mt (The Buck Institute for Education, 2009); "How to Develop aStandards-Based Unit of Study” (Kentucky Department Fae) 
of Education, 2007); and High Schools That Wark ond Making Midale Grodes Work Summer Institute (Southern Regional Education Board, 2008. ss 
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Key Evaluation Design Features 


The evaluation is designed to measure the impact of the targeted intervention on student 
outcomes teacher efficacy. The impact study was guided by the following research questions: 


The impact research questions that motivated the study design and analysis plan are: 


e What is the impact of Kentucky’s Cognitive Literacy Model (KCLM) on the reading 
achievement of low achieving readers? 

e What is the impact of KCLM on the writing achievement of low achieving readers? 

e What is the impact of KCLM on the perceived reading strategy use of low achieving 
readers? 

e What is the impact of KCLM on the motivation and engagement of low achieving readers? 


Student outcome measures are as follows: 


e Group Reading Assessment and Diagnostic Evaluation (GRADE) 

e The Kentucky State Writing Assessment 

e Metacognitive Awareness of Reading Strategies Inventory (MARSI) 
e Student Motivation Survey 


In addition, the following implementation questions were answered: 


e What is the state-level implementation of the professional development training and 
support for intervention teachers in the project? 

e Which components of KCLM were implemented most frequently by teachers in 
classrooms? 

e What was the quality of KCLM implementation in classrooms? 


Evaluation of Implementation 
Summary of the Design of the Implementation Study 
The research questions that guided the implementation study of the targeted intervention are: 
e What is the state-level implementation of the professional development training and 
support for intervention teachers in the project? 
e Which components of KCLM were implemented most frequently by teachers in 


classrooms? 
e What was the quality of KCLM implementation in classrooms? 
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Implementation Data Collection and Analysis 


Summer training. During the summer of 2010, the selected intervention teachers from 
each school received five days of initial training on the core strands. This training was developed 
and provided by the Kentucky Department of Education literacy consultants. Attendance records 
were kept at each training session, and individual teacher attendance was computed at the end of 
the training in terms of percentage of days attended. School administrators will be required to 
attend one day of summer training to learn about the goals of the intervention and the 
expectations for support of the interventionist and students. Participation will be assessed and 
scored for adequacy and fidelity through attendance records provided by the developers. 


Research assistants attended each training session and took detailed field notes in five 
minute intervals. A code list was developed related to Content and Delivery. Content codes 
related to the key components of KCLM. Delivery codes related to various training formats such as 
whole group, small group, discussion, and lecture. To establish reliability, three research assistants 
independently coded one day of field notes. Agreement was 90% for content and 90% for delivery. 
After the coding the research assistants agreed that some of the codes should be further broken 
down for a more accurate description of what was going on during the time interval. Research 
assistants and investigators discussed adding additional codes for content and delivery. The 
following codes were added to content: Collaboration, Planning-working, and Planning-discussion. 
The following codes were added to delivery: Whole group-lecture and Whole group- discussion. 
The research assistants continued independently coding the next four days of the intervention 
teacher training. Research assistants met again to check agreement. Agreement achieved for 
content was 95% and 100% agreement for delivery. 


Coaching and mentoring. Interventionists received on-going training and support through 
an online learning community that included book studies, Webinars, a discussion forum, and 
repositories for sharing resources. KDE consultants made site visits to the schools in the fall and 
again in the spring to provide on-site coaching and support. Participation was assessed and scored 
for adequacy and fidelity through attendance records provided by the developers. 


Classroom instruction. Implementation fidelity for the treatment condition was 
established through classroom observations using a standardized observation protocol for 
intervention classes. Reading intervention teachers were observed twice during the year. The 
standardized observation protocol for intervention teachers included two components: a checklist 
of essential KCLM components and a quality rubric for assessing teachers’ implementation quality. 
In the fall 2010, two research assistants met to create a list of model components based on the 
trainings from KDE. This list was sent to KDE for validation. KDE added a few components to make 
the list complete. To identify the model features that were most critical to program success, 
evaluators asked the developers to rank each set of features for each component in order of 
importance. To confirm these rankings, evaluators sought feedback from three expert scholars 
who confirmed and elaborated on the shortened list for each feature. The lead evaluator and 
KCLM trainer viewed two videos of KCLM instruction and independently completed the protocol. 
Then, they discussed their scores, reached consensus on scoring, and made minor revisions to 
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some wording on the protocol. For training in using the protocol, research assistants viewed one 
video, rated the instruction using the protocol, and discussed disagreements. To establish inter- 
observer agreement, the research assistants viewed the second video and independently 
completed a protocol. Agreement with the lead evaluator on the checklist of key aspects of KCLM 
components for this protocol was 74.4%. The largest area of disagreement was related to 
vocabulary instruction under the “Instructional Strategies” component. The lead evaluator 
provided additional training around vocabulary instruction to clarify misunderstandings. 
Agreement on the quality descriptors for the second video was 91.7%. 


It was expected that teachers would implement some aspect of each for the four model 
components (motivation and engagement, strategic processes, instructional strategies, and 
communication) during each class period but it was not expected that teachers would implement 
every aspect of each component each class period. The protocol yielded information on which 
aspects of each component were implemented during observations and whether the quality of 
implementation was (a) developing, (b) adequate, or (c) exemplary. Percentages of observations 
that included each component were computed, and the proportion of observations rated at each 
quality level was provided. 


KCLM Implementation Results 
Characteristics of Interventionists 


Nine interventionist positions were filled by 11 teachers during the year. Intervention 
teachers were selected and hired by individual school districts and met the planned 
characteristics, according to school personnel. A total of nine interventionists were hired in the 
summer of 2010. During the course of the school year, two interventionists left and were 
replaced, for a total of 11 interventionists. These interventionists implemented the KCLM targeted 
interventions within the nine schools. One interventionist was male and 10 were female. All 
interventionists were white. All of the interventionists had a masters degree or higher, and three 
(27%) were certified as reading specialists. Interventionists had an average of 12.9 years of 
experience. 


Implementation of Professional Development Model 
Interventionist training. Training was provided to interventionists for five days in the 


summer, and three days during the school year. Table 2.1 shows the content amount and delivery 
of the professional development model. 
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Table 2.1 


Content and delivery of professional development training (KCLM) 


Number of Percent of training 
Minutes total 
Content Housekeeping 70 2.77 
Overview of grant 220 8.71 
Collaboration 275 10.89 
Reading strategies & Strategic processing 640 25.30 
Communication 175 6.93 
Motivation and behavior management 225 8.91 
Foundational reading — basic skills 60 2.38 
Assessment 360 14.26 
Planning work 280 11.09 
Planning discussion 115 4.55 
Project based learning 110 4.36 
Delivery Whole group lecture 790 31.23 
Format Whole group discussion 1035 40.91 
Small group 340 13.44 
Individual 365 14.43 


Professional development inputs. During the summer training, seven of the nine 
interventionists attended all five days. One interventionist attended four days and was absent 
one day due to illness. This material was covered with the absentee during the following day’s 
training during lunch and breaks. Another interventionist did not attend the summer training. 
That interventionist was trained one-on-one during three days at the interventionist’s school. 
During the school year there were three regular training dates. All interventionists attended two 
of these dates, and eight attended the third. The absent interventionist made up the missed 
training date via Skype with KDE staff. Interventionists also attended the Kentucky Reading 
Association conference. All nine interventionists attended all three days of the conference. 
Attendance and participation was adequate for all interventionists. 


In addition to summer training, interventionists received site visit support from KDE staff 
throughout the year. The number of visits ranged from four days to 11 days per interventionist at 
six hours per day with an average of 5.56 days. When extra support was needed, additional visits 
were scheduled. Items discussed during the visits were strategy implementation, project-based 
learning, data collection, student engagement and motivation, assessment procedures and other 
various topics depending on type of support needed related to the KCLM model. 


Interventionists received support through the use of a NING, Skype, email and phone calls. 
Phone calls, email and Skype were used for ongoing support and professional development. The 
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NING was used for book studies, discussion forums, blogging and sharing resources. KDE staff 
estimate that each interventionist was supported an estimated three times per week via phone 
calls and emails, an estimated once per week via NING, and once per month via Skype. 


All participation in the professional development training and support was considered to 
be adequate for all interventionists. 


Professional development for administrators. Administrators were provided with one 
training day in the summer of 2010 to discuss the requirements of the grant and ways they could 
support the intervention. Administrators from all nine schools attended. 


During the school year, KDE staff met with school administrators to discuss grant 
requirements, evidence-based components of the targeted intervention, scheduling issues, 
observations of interventionists, literacy leadership, literacy planning, and meeting the needs of 
adolescent learners—especially low achieving students. The number of meetings ranged from two 
to six per school with an average of 3.11 meetings per school. KDE staff also shared a KCLM 
Update newsletter with all administrators four times throughout the year. All administrator 
participation was considered to be adequate. 


Implementation of Classroom Model 


Class size, intensity, and duration. Class sizes varied throughout the year due to student 
attrition (i.e., transferring schools, dropping out, etc.). Classes ranged from as few as 12 students 
to as many as 20. The intervention class met every day throughout the course of the year, and 
ranged from 45 minutes to 75 minutes daily. 


Classroom Implementation Results. The observation protocol included ratings for both 
presence of indicators for each component for KCLM, and quality of each component of KCLM. 
The next four tables show the number of times interventionists were observed incorporating 
indicators of motivation and engagement, strategic processing, instructional strategies, and 
communication skills in their lessons. Table 2.2 indicates the model components most widely 
implemented by teachers and the numbers and percents of lessons that included those 
components. Other model components were observed in fewer than 50% of lessons. 
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Most Widely Implemented Model Components 


Model Component Round 1 % Round 2 % 
N=18 N=18 
Motivation & Engagement 
Teacher makes connections between 13 7 9 50 
learning activities and real world issues 
Teacher varies instructional format, i.e. 9 50 9 50 
group work, lecture, and partner work 
Student prior knowledge, interest and 11 61 2 67 
background used in determining content 
Strategic Processing 
Teacher explains cognitive strategies for 14 78 9 50 
comprehension 
Teacher models using cognitive strategies 2 67 10 56 


for comprehension 

Teacher encourages and provides 

opportunities for students to practice using 15 83 13 72 

cognitive strategies for comprehension 
Instructional Strategies 

Teacher explicitly incorporates higher level 


; 7 39 10 56 
questions 
Teacher uses instructional tools to support 12 67 6 33 
student comprehension or vocabulary 
Teacher describes, explains, and provides 3 eT 11 61 
an example of new key terms 

Communication Skills 

Teacher uses during text-based discussion 11 61 6 33 
strategies 
Teacher includes writing to learn activities 15 83 9 50 


For each KCLM component, interventionists focused on some components more than 
others. For motivation and engagement, interventionists were more likely to make connections 
between the lesson and real world issues, to vary instructional format, and to tie the content into 
student experiences. Interventionists were not as likely to incorporate technology, provide 
student feedback, facilitate problem solving and provide students with choices. For strategic 
processing, interventionists were more likely to explain, model and encourage student practice of 
cognitive strategies. Interventionists were not as likely to ask student to reflect on cognitive 
processing and reading comprehension or provide explicit instruction in foundational reading 
skills. Overall, interventionists were not as likely to incorporate instructional strategies into their 
lessons. In the first round of observations, interventionists did use instructional tools to support 
student comprehension or vocabulary, and in the second round they were likely to describe, 
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explain and give examples of new vocabulary words. Communication skills were also under- 
utilized by interventionists. There was some evidence of teacher uses during text-based 
discussion strategies, and writing to learn activities. 


Classroom observations also included a quality scoring for each KCLM component. Table 
2.3 shows each teacher’s quality score by KCLM component for each round of observation. Each 
round represents an average between scores for each class period observed. Table 2.3 shows the 
mean scores by teacher for quality of each KCLM component and the overall score for the model. 
In terms of overall quality of implementation, implementation of KCLM was adequate in 3 of the 9 
classrooms. 


Table 2.3 


Mean Scores by Teacher for Quality of KCLM component 


Motivation Strategic Instructional Communication Overall 
Classrooms f . ; 
Engagement Processing Strategies Skills score 
Mean Mean Mean Mean Mean 
1 1 1 1 1 1 
2 1.5 1.5 1 1.5 1.38 
3 2.25 2 2 2 2.06 
4 1 1 1 1 1 
5 2 2 2 1.5 1.88 
6 2 2 2 1.5 1.88 
7 1 1 1 1 1 
8 1.5 1 1 1 1.13 
9 1 1 1 1 1 


Note. 1-1.4=Developing, 1.5-2=Adequate, 2.1-3=Exemplary 


Experiences for control students during intervention period. Students who were selected 
for the control group received a regular elective as part of their freshman program. A wide 
range of electives were taken including band, chorus, civics, and physical education. 


Additional reading programs. In two schools, additional reading assistance programs were 
provided to students who qualified. At one school students from both the intervention and 
control groups took an unstructured reading class where the teacher had access to materials 
from Study Island and Discovery Education as well as Read 180 and System 44. In the second 
school, students in both intervention and control groups took a one semester class called 
Reading Revisited, a highly structured class relying heavily on vocabulary workbook exercises. 


Implications for impact analysis. Two factors related to implementation should be 


considered when interpreting the impact analysis to follow. First, teachers were learning to 
implement the intervention as they were implementing it, and this resulted in a variation in 
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implementation quality. Teachers were adhering to the major components of the intervention, 
but they did not necessarily implement those components as they were designed to be 
implemented. Therefore, impact findings should not be directly attributed to the intervention 
model as it is designed. Second, that some schools continued to implement reading programs 
targeted at low achieving readers presents a confounding factor that may have influenced 
outcomes for both treatment and control groups. 


Evaluation of Impact 
Study Design 


Sampling selection process. Evaluators implemented a stratified random sampling 
procedure for students within each school using four demographic variables: special education 
status, free/reduced lunch status, ethnicity, and gender. Within each school students were 
sorted by demographic variables creating subgroups of students, and students within each 
subgroup were then sorted by assessment score. Using a random number generator to assign 
the first student to either the intervention or control group, the students were then alternately 
assigned sequentially to the intervention or control group. 


Evaluators provided a list of students that qualified for the intervention group to the nine 
participating high schools during the summer of 2011. These students were scheduled to be in 
the intervention class, and students and parents were notified when the class schedules were 
given to all students at the beginning of the fall semester. Schools were not provided the 
names of the students in the control group. 


A student was removed from the study post-random assignment if he/she dropped out of 
school or moved/transferred to a school not participating in the Striving Readers program. 
Also, if the student did not take the posttest in the spring of gin grade, they were removed. 
Finally, a student was ineligible post-random assignment if, after the student enrolled in the 
ninth-grade, the high school places the student in all day resource classes. The schools were 
instructed to inform the evaluators at the end of the study if a control student was placed in all 
day resource classes. Thus the criterion the high school used to assign a student to all day 
resource would not be affected by the results of the random assignment, and was applied 
equally to intervention and control students. There were 13 intervention students that were 
assigned to all day resource classes post-random assignment, and are ineligible to be in the 
study. There were sixteen control students who were placed in all day resource classes after 
random assignment. 


Sample size. Exhibit 1 shows the sample size results for reading achievement test, 
GRADE, divided by condition. Out of the population of 2,229 eighth grade students, 637 were 
identified as low achieving readers (NCE of 40 or lower). Three hundred nineteen students 
were assigned to the treatment group, and 318 students were assigned to the control group. 
After random assignment, 13 students in the treatment group and 16 students in the control 
group were assigned to all day resource classes after enrolling in high school so were ineligible 
for the study. Additionally, 22 students in the treatment group did not enroll in a participating 
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high school. The number of students in the control group that did not enroll in a participating 
high school is not available. 


There were 306 targeted, eligible students at baseline in the intervention group and 302 
students in the control group. Attrition in the intervention group at the end of the school year 
totaled 74 students (36 students moved and 38 students did not take the spring test), resulting 
in an analytic sample of 232 intervention students. Attrition in the control group totaled 49 
students (34 students either did not enroll in the participating high school or moved during the 
year and 15 students did not take the spring test), resulting in an analytic sample of 253 control 
students. 


Exhibit 1: Consort Chart for Kentucky: GRADE 


POPULATION OF STUDENTS 


N= 2229 


ELIGIBLE STUDENTS INELIGIBLE STUDENTS 
n= 637 n= 1692 


RANDOM ASSIGNEMNT 


TREATMENT GROUP CONTROL GROUP 


n= 318 


| | 


Not Eligible for Study Post-RA Eligible for Treatment Eligible for Control Not Eligible for Study PostRA 
n= 13 n= 306 n= 302 n= 16 


n= 


Attrition Post-RA 
n= 


Attrition at Follow-up 
n= 

Attrition due to: 

* Moved 

* Unable to locate 
*No test scores 

* Other 


| 


Targeted Sample 
at Baseline 
n= 


Final Intent-to-Treat 
Analytic Sample at 


Follow-up 


n= 


Targeted Sample 
at Baseline 
n= 


Final Intent-to-Treat 
Analytic Sample at 
Follow-up 


n= 


Attrition PostRA 
n= N/A 


Attrition at Follow-up 
n= 49 
Attrition due to: 

* Moved 

* Unable to locate 

* No test scores 

* Other 
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Exhibit 2 shows the sample size results for the writing achievement, by condition. The 
targeted number of eligible students at baseline in the intervention group was 306, with 302 
students in the control group. Attrition in the intervention group during the school year totaled 
134students (36 students moved and 98 students did not take the spring writing test), resulting 
in an analytic sample of 172 intervention students. Attrition in the control group totaled 109 
students (34 students either did not enroll in the participating high school or moved during the 
year, and 75 students did not complete the spring survey), resulting in an analytic sample of 
193 control student. 


Exhibit 2: Consort Chart for Kentucky: Writing 


POPULATION OF STUDENTS 


N= 2229 


f * 


ELIGIBLE STUDENTS INELIGIBLE STUDENTS 
n= 637 n= 1692 


RANDOM ASSIGNEMENT 


TREATMENT GROUP CONTROL GROUP 


n= n= 318 


| { 


Not Eligible for Study Post-RA Eligible for Treatrnent Eligible for Control Not Eligible for Study Post-RA 
n= 13 n= 306 n= 302 n= 16 


Attrition Post-RA | | 


n= 


Attrition PostRA 
n= N/A 


Targeted Sample 
at Baseline 


Targeted Sample 
at Baseline 


Attrition at Follow-up 


n= 134 
Attrition due to: 

* Moved 

* Unable to locate 
* No test scores 

* Other 


n= 


Final Intent-to- Treat 
Analytic Sample at 
Follow-up 


n= 


hic 


Final Intent-to-Treat 
Analytic Sample at 
Follow-up 


n= 


Attrition at Follow-up 
n= 109 
Attrition due to: 

* Moved 

* Unable to locate 

* No test scores 

* Other 
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Exhibit 3 shows the sample size results for student survey, by condition. The targeted 
number of eligible students at baseline in the intervention group was 306, with 302 students in 
the control group. Attrition in the intervention group during the school year totaled 152 
students (36 students moved and 116 students did not complete the spring survey), resulting in 
an analytic sample of 154 intervention students. Attrition in the control group totaled 129 
students (34 students either did not enroll in the participating high school or moved during the 
year, and 95 students did not complete the spring survey), resulting in an analytic sample of 
173 control student 


Exhibit 3: Consort Chart for Kentucky: SE Motiv 


POPULATION OF STUDENTS 


N= 2229 


ELIGIBLE STUDENTS INELIGIBLE STUDENTS 
n= 637 n= 1692 
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TREATMENT GROUP 


n= 


Not Eligible for Study Post-RA 
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Attrition at Follow-up 
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* Moved 

* Unable to locate 
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Impact Measures and Data Collection 


The following measures were used to ascertain the impact of the KCLM intervention on 
students’ reading and writing achievement, reading strategy use, and reading motivation. 


Group Reading and Diagnostic Evaluation (GRADE). The GRADE is a nationally normed 
reading assessment, and it includes normalized scores for overall reading achievement. Normal 
Curve Equivalent scores (NCEs) including vocabulary and comprehension items closely align 
student outcomes with the goals of the intervention. In the spring of the planning year, the 
GRADE (level M) was given to all eighth-grade students in participating feeder schools as a 
pretest. In the following spring, the GRADE (level H) was given to all ninth-grade students as a 
postest. 


Kentucky State Writing Assessment. The Kentucky state writing assessment provides a 
holistic score based on analytic categories which closely align student outcomes with the goals 
of the intervention. The State Writing Assessment was given to all eighth-grade students in 
spring of the planning year, and all ninth-grade students in spring of the following year. The 
test is designed so that students get a choice between two writing tasks which include three 
possible modes of writing (inform, narrate for a purpose, or persuade) and four possible 
response formats (article, editorial, letter or speech). In addition to a writing task, each student 
is also presented with draft versions of three pieces of writing. Four multiple-choice questions 
dealing with editing/revising are provided with each of the three drafts so, each student also 
responds to 12 multiple-choice questions. 


Metacognitive Awareness of Reading Strategies Inventory (MARSI). The MARSI (Mokhtari 
& Reichard, 2002) is a student self-report measure designed specifically to assess adolescents’ 
perceived use of reading strategies during academic reading. For this study, the MARSI was 
adapted to measure self-efficacy with strategy use. “I can” was added to the beginning of each 
strategy statement in the survey. The survey was given to all students in fall and spring of ninth 
grade. The survey items are presented on a scale of 1 to 5, where 1 is equal to “not at all 
confident” and 5 is equal to “completely confident.” Example items would be “I am able to 
have a purpose in mind while | read” or “I can take notes while reading to help me understand 
what | read.” 


Adolescent Motivation Survey. In the spring of eighth grade and in the spring of ninth 
grade, all students in participating schools (and eighth-grade feeder schools) completed a 
survey measuring several dimensions of intrinsic reading motivation (challenge, curiosity, 
intrinsic task value, attainment), extrinsic reading motivation (extrinsic task value, compliance), 
reading related self-beliefs (expectancy, difficulty) and leisure reading. The survey items are 
presented on a scale of 1 to 5 and the value varies depending on the dimension measured. A 
sample item for the challenge dimension is “I like hard, challenging books” where 1 equals “not 
at all true and 5 equals “very true.” A sample item from the leisure dimension is “How much 
time have you spent reading a magazine this week?” with 1 equaling “none,” and 5 equaling 
“more than five hours.” Reliability for subscales were >.70 with the exception of leisure 
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(a=.585) and extrinsic task value (a=.60). These scales were taken from existing measures 
(Eccles & Wigfield, 1995; Hopper, 2005; Wigfield & Guthrie, 1997). 


Summary of Analytic Approach 


Hierarchical Linear Models (HLMs) were used to estimate the impact of the KCLM on 
student achievement, motivation, and reading strategies outcomes. The GRADE Normal Curve 
Equivalents (NCEs) were used to estimate the impact of the KCLM intervention on achievement. 
Holistic scores from the state writing assessment were used to estimate the impact of the KCLM 
on writing achievement. The average of the items on the MARSI was used to estimate the 
impact on reading strategy use, and the average of the items on the Adolescent Motivation 
Survey was used to estimate the impact on motivation. 


A two-level HLM model (students assigned to intervention or control group within schools) 
will be used to determine the impact of KCLM. Four hypotheses will be tested: 
H1: The KCLM intervention has no impact on student achievement. 
H2: The KCLM intervention has no impact on student writing achievement. 
H2: The KCLM intervention has no impact on self efficacy of student strategy use. 
H3: The KCLM intervention has no impact on student motivation. 


Level-1 HLM: Student Level. At the student level, the spring outcome variable (reading 
achievement, writing achievement, strategy use, or motivation) will be modeled as a function 
of fall outcome variables (covariate), intervention/control status and four demographic 
variables: gender, ethnicity, free/reduced lunch status, and special education. 


Level-2 HLM: School Level. This analysis will be performed on ninth-grade students’ scores 
from nine high schools. In addition to the base year Reading KCCT score, other school level 
variables that will be included are the school percent of students qualifying for free or reduced 
lunch fees, school percent of white students in the school, and school percent of black 
students, and the percent of students with disabilities. 


Description of the First Year Sample 


Nine high schools geographically distributed across the state participated in the study. 
High school demographic data was collected from the Kentucky Department of Education 
website for the 2009-10 academic year. The average number of 1o"-42" grade students 
enrolled at the participating schools was 454, ranging from 301 to 803 students. The average 
percent of White students was 91.58% (53.9%, 98.8%), and the average percent Black students 
was 5.76% (0%, 37.7%). The average percent of students receiving free/reduced lunch was 
62.25% (40.7%, 83.5%), and the average number of students enrolled in special education 
classes was 12.73%, ranging from 7.2% to 16.5%. 


Student demographic data was collected from the middle schools for every enrolled 
eighth-grade student. Schools were contacted for student demographic information if there 
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were students that were not on these lists but completed the eighth-grade. Table 2.4 shows 
demographic characteristics of students in the intervention and control groups. Students in the 
Striving Readers classes were typically White males, receiving free/reduced lunch services and 
are not assigned to special education classes. Students that received the intervention were 
very similar in demographics as compared to the control group, with the possible exception of 
gender, where a slightly higher percent of males received the intervention. 


Table 2.4 


Intervention and Control Student Demographics (and Proportions) 


Gender Ethnicity Lunch Special Education 
Group Male Female White Minor- Reg Free/ Not Special 
ity Pay Red In Ed. 

Interv 136 96 205 27 40 192 161 71 

(.59) (.41) (.88) (.12) (.17) (.83) (.69) (.31) 
Contl 140 113 219 34 45 208 179 74 

(.55) (.45) (.87) (.13) (.18) (.82) (.71) (.29) 
Total 276 109 424 61 85 400 340 145 

(.57) (.43) (.88) (.12) (.38) (.62) (.73) (.16) 


Impacts on Students 


Table 2.5 below shows the results on student reading achievement, writing 
achievement, reading strategy use self efficacy and reading motivation for intervention and 
control students after one year of intervention. The unadjusted means and standard deviations 
for each measure is displayed, and the means adjusted for the HLM results are displayed. The 
estimated impact of the intervention, the effect size, and the significance level are shown. 
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Table 2.5 


Impact of the Target Intervention on Student Reading Achievement, Writing Achievement, 
Strategy Use Self Efficacy, and Motivation 


Unadjusted HLM-adjusted 
Means Means 
Control Tx Control Tx Estimated Effect p 


Impact Size 
Reading Achievement 


Spring NCE 36.7 35.6 37.2 36.4 -0.79 -0.059 .439 
(13.49) (13.48) 
No. of students 253 232 
Writing Achievement 
Spring score 826.3 827.2 826.9 827.6 0.62 0.066 481 
(9.41) (11.78) 
No. of students 193 172 


Strategy Use Self Efficacy 


Spring score 3.3 3.5 3.4 3.5 0.154 0.250 .012* 
(0.62) (0.61) 
No. of students 173 154 
Motivation 
Spring score 2.9 3.0 2.9 3.0 128 0.217 .015* 
(0.59) (0.62) 
No. of students 173 154 


Note. Standard deviations are presented in parenthesis. Effect size calculated as the impact 
divided by the control group standard deviation. 
*Designates statistical significance at the .05 level of significance. 


There is no significant effect of the intervention at the .05 level on the reading and 


writing achievement after the first year of the program. However, a significant effect of the 
intervention is shown for strategy use self efficacy and reading motivation. 
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Discussion and Conclusions 


The impact results from one year of study do not reveal significant effects of the KCLM 
intervention on students’ reading or writing achievement, but there were significant impacts on 
students’ self-efficacy for strategy use and students’ reading motivation. Development of 
demonstrable improvements in reading performance may require greater lengths of time to 
gain comfort with flexible strategy use and to reap benefits of increased motivation. While it 
seems the students in this study reported increased confidence with using reading strategies, 
they may not have internalized and practiced strategy use to a sufficient enough extent to 
achieve purposeful flexible use under a wide range of conditions. Nevertheless, the impacts of 
the intervention on students’ strategy use and motivation are noteworthy given the emphasis 
placed on these dimensions of learning in recommendations for improving adolescents’ literacy 
achievement (Biancarosa & Snow, 2006; Kamil, Borman, Dole, Kral, Salinger, & Torgesen, 2008). 


Although the planned professional development model was implemented at high levels, 
implementation of the classroom model was lower. Elements of the KCLM model were evident 
in each classroom observation, but the teachers in this project implemented some components 
of the intervention to a greater extent than other important components. Also, the majority of 
teachers implemented the intervention at developing levels of quality. Higher levels of 
implementation may have resulted in higher levels of achievement for students. It is probable 
that teachers would have been able to achieve higher levels of implementation in future years 
with ongoing support had the project extended for the full planned project duration. 
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Appendix A 
Summary of Analytic Approach 


Hierarchical Linear Models (HLMs) were used to estimate the impact of the intervention 
on student achievement in reading and writing, and motivation, and self-efficacy outcomes. 
The GRADE Normal Curve Equivalents (NCEs) were used to estimate the impact of the 
intervention on reading achievement, and the standard writing scores were used to measure 
the impact on writing. The average survey scores were used to estimate the impact on self- 
efficacy in reading strategy use and impact on motivation. 


A two-level HLM model (students assigned to intervention or control group within 
schools) was used to determine the impact of the targeted intervention. At the student level, 
the spring outcome variable (reading achievement, writing achievement, self-efficacy in 
strategy use, or motivation) was modeled as a function of fall outcome variables, 
intervention/control status, and four demographic variables: gender, ethnicity, free/reduced 
lunch status, and special education. 


Level-1 Model: Student Outcomes (achievement, reading strategies, or motivation) 
M 
Yi= Bo + BalY¥*i) + Ba (Ty) + YB mi A mij + € ij 
m=3 


where 


Y, is the spring student outcome (post-test) score for student i at school j; 

PB ois the mean student outcome (post-test) score for control students at school j; 

Y*is the fall student outcome (pre-test) score for student i centered at school j; 

Pf 1;is the average student outcome (pre-test) slope for students at school j; 

Tj = 1 if student i is assigned to LSC intervention at school j, and 0 if control; 

£2, is the mean difference of student outcome pre-post gain between intervention and 
control students at school j; 

Q@ mij are additional covariates representing demographic characteristics of student i at 
school j (gender, ethnicity, free/reduced lunch, and special education status); 

PB mare coefficients corresponding to student demographic covariates (gender, ethnicity, 
free/reduced lunch, special education status), and 

€ ; is the random effect representing the difference between student ij’s score and the 
predicted mean score for school j. These residual effects are assumed normally distributed 
with mean 0 and variance o?. 


Level-2 Model: Student Achievement — School Level 
This analysis was performed on data from 9" grade students collected for one year. The 


covariates in this model pertain to the concurrent year the student was in the intervention or 
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control group with the exception of the Reading Kentucky Core Content Test (KCCT) score, for 
which the score for the base year, spring, 2010, was used. In addition to the base year Reading 
KCCT score, other school level covariates included enrollment, percent of white students in the 
school, percent of African American students, percent of students qualifying for free or reduced 
lunch fees and percent of students with disabilities.. 


Q 
Bo = Yoot YY ogWa + H oj 
q 


Bij= 710 
Ba> 71 
B m= Y¥mo 
where 


Yoo is the mean student outcome (post-test) score of gin grade control students in 
Kentucky Striving Readers middle schools 

W,) are school level covariates including base year Reading KCCT (spring, 2010), and 
average school percent free/reduced lunch, percent white students, percent black students, 
and percent disability; 

Y oq are coefficients corresponding to school-level covariates; 

Lo is the unique effect of school j on mean student outcome, holding W,; constant (or 
conditioning on W,;) - this effect is assumed normally distributed with mean 0 and variance 7’; 

7 10 is the fall student outcome (pre-test) slope; 

¥ 20 is the overall target intervention treatment effect on spring student outcome (post- 
test) scores; 

vy mo is the fixed m'" student covariate effect (gender, ethnicity, free/reduced lunch, special 
education status) on the spring outcome variable. 


Selection of Covariates. The random assignment procedure included all student 


demographic variables in the HLM model, so were included regardless of significance. 
Interaction effects were not considered. 
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Appendix B 
Study Measures 
Student Survey 


Items for the student survey were adapted from the following pre-existing inventories: 


Eccles, J. S., & Wigfield, A. (1995). In the mind of the actor: The structure of adolescents’ 
achievement task values and expectancy-related beliefs. Personality and Social 
Psychology Bulletin, 21(3), 215-225. doi: 10.1177/0146167295213003 

Hopper, R. (2005). What are teenagers reading? Adolescent fiction reading habits and reading 
choices. Literacy 39(3), 113-120. doi: 10.1111/j.1467-9345.2005.00409.xc 

Mokhtari, K. & Reichard, C. A. (2002). Assessing students‘ metacognitive awareness of 
reading strategies. Journal of Educational Psychology, 94(2), 249-259. doi: 
10.1037/0022-0663.94.2.249 

Wigfield, A., & Guthrie, J. T. (1997). Relations of children’s motivation for reading to the amount 
and breadth of their reading. Journal of Educational Psychology, 89(3), 420-432. doi: 
10.1037/0022-0663.89.3.420 
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This survey contains statements about what people can do when they read academic or school-related materials 
to help them understand what they read. Answer each question as honestly and truthfully as you can, there are no 
right or wrong answers. Please completely BUBBLE In each answer. 


Your answers on this survey are completely confidential. Information will be released in summary form only. 


Please BUBBLE the number that shows how CONFIDENT you are In each item, using the following scale: 


1 2 3 4 5 
Not at ali Confident Completely Confident 
1.lam able to have a purpose in mind when | read............cccccccseesesesceeesees 10) ® © @ ® 
2. | can take notes while reading to help me understand what I read.......... © @ ©) @ © 
3. | can think about what | know to help me understand what | read............ ® ® ® © © 
4. | can preview the text to see what it's about before reading it................. 10) ® © © © 
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1 2 3 4 § 
Not at all Confident Completely Confident 
5. When text becomes difficult, | can read aloud to help ME... 0) ® © ® © 


understand what | read. 


6. | can summarize what | read to reflect on important information............. © 0 © @® © 
in the text 
7. | can think about whether the content of the text fits MY........esserseeeees Oo © 0 6 © 


reading purpose 


8. | can read slowly but carefully to be sure | understand what I'm reading Q @ 6) ® © 


9. | can discuss what I read with others to check my understanding........... 0) ® © ® © 
10. | can skim the text first by noting characteristics like length and........... 0) @ © ® ® 
organization 

11. | can get back on track when | lose Concent ation........ssecesscesesesseneees 10} ® @ © © 


42. | can underline or circle information in the text to help me remember it 10} ® @ 1) ® 


13. | can adjust my reading speed according to what I'm reading.............+. ® 1) @ © ® 
14. | can make good decisions about what to read closely and what to © © © © © 
ignore 

15. | can use reference materials such as dictionaries to help mé............: Oo © @ ® © 


understand what I read. 


16. When text becomes difficult, | can pay closer attention to..........ss0 Oo © © ® © 
what I'm reading 
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1 ; 2 3 4 5 
Not at all Confident Completely Confident 


17. | can use tables, figures, and pictures in text to increase my............... (0) @) 6) @ 
understanding 


18. | can stop from time to time and think about what I'm reading................ 0) ® © © 


19. | can use context clues to help me better understand what I'm reading 1} o @ 


20. | can paraphrase (restate ideas in my own words) to better..........000+ © © @) ® 
understand what | read. 


21. | can picture or visualize information to help remember what | read...... ® 12) ® ® 


22. | can use typographical aids like boldface, and italics to identify.......... 0) (0) o) ® 
key information. 


23. | can critically analyze and evaluate the information presented............ 0) 0) © ® 
in the text 

24. | can go back and forth in the text to find relationships among............+. © © © @ 
ideas in it 

25. | can check my understanding when | come across conflicting............. ® (a) © ® 
information 

26. | can predict what the material is about when I read..........csesssssesesssesree 10) ® ® © 


27. When text becomes difficult, | can reread to increase my understanding 10) @ @ o) 
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Please BUBBLE the number that shows how TRUE each item is for you, using the following scale: 


Not at al True ; a a True . ware True 

28. | can ask myself questions ! would like to have answered in the text... ® ® ©) © © 
29. | can check to see if my guesses about the text are right or wrong...... ® ® @ ® © . 
30. | can guess the meaning of unknown words OF Phrases.........seseseeeeee ® ® @ © © 
31. 1 like hard, Challenging DOOKS.........:sssssessesseeesessssteeeeseereereteesseaseeeese 10) ® © © © 
32. If the project is interesting, | can read difficult material.............seseree ® ®@ 6) ® ® 
33. | like it when the questions in books make me think............:..:+ceseseee o 0 0 6 ® 
34. | don't like it when we get a lot of difficult reAdING..........csscseeerserereeerees 10) ® ® © ©) 
35. | usually learn difficult things by reading... mien ® @ @ © © 
36. If a book is interesting | don't care how hard it is to read.............s000 10) ® © © ® 
37. If the teacher discusses something interesting | might read..............-.. 0) ® © ® © 
more about it 

38. I have favorite subjects that I like to read ADOU........scessssesssesesssseeeeees o 0 0 © ® 
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A 3 
Not at all True Somewhat True 


39. | read to learn new information about topics that interest me...... 


40. | read about my hobbies to learn more about them..........:000000 


41. I like to read about new thingS..........ccccsssscsesessseseseseeesterseseeces 


42. | do as little schoolwork as possible in reading.............ccecceeeee 


43. | read because | HAVE tO........c.cccccsessesseeserseescscescceceereecensenseeseanes 


44. | read things that are not assigned................c.ccccecccesceseseseeeeseeeee 


45. | always do my reading work exactly as the teacher wants it...... 


46, Finishing every reading assignment is very important to me....... 


47. | always try to finish my reading On time.............cceseeseeeeeseeeeeerers 


48. | do schoolwork so that the teacher can make sure | am paying 


attention 


5 
Very True 


seenennees 
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Here are some questions about YOURSELF AS A STUDENT. Notice that the choices are under 
each question. 


49. How well do you think you will do in reading this year? 
(1=not at all well, 3=somewhat well, S=very Well)...ccssssesesseseeee 


50. In general, how hard is reading fOr YOU?........ss.sssscsesessseseeeeseneeeeeteres © (0) © ® © 
(1=not at all hard, 3=somewhat hard, 5=very hard) 


51. In general, | find working on reading assignments 
(1=not at all interesting, 3=somewhat interesting, 5=very interesting) 


© 
© 
© 
© 
© 


52. Is the amount of effort it will take to do well in reading worthwhile to you? 
(1=not at all worthwhile, 3=somewhat worthwhile, 5=very worthwhile) 


© 
2) 
© 
© 
© 


53. How useful is periee for what you want to do after you graduate and 
go to work? (1=not at all useful, 3=somewhat useful, 5=very useful) 0) a) © @ © 


54. How good at reading are YOU? ........ccscsssessesesesesessessesssesesesensesseeseseeses 
(1=not at all good, 3=somewhat good, 5=very good) © ® © @ © 


55. Compared to most other students in your class, how hard is reading 
for yout1= not at all harder, Sescramatint harder, 5= much harder) Oo) ® © © © 


56. How much do you like reading? 
(1=don't like k at all, Hccealial like it, 5=like it very much)....... ® ® ® @ © 


57. | feel that, for me, being good at reading is: 
(1 = not at all important, 3=somewhat important, 5 = very important) ® ® © ® © 
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Here are some questions about YOURSELF AS A STUDENT. Notice that the choices are under 
each question. 


58. How useful is reading for your daily life outside school? 
(1=not at all useful, 3=somewhat useful, 5=very Useful)... @ @ © 10) © 


59. How have you been doing in reading this year? 
(1=not at all well, 3=somewhat well, 5=very Well)...cscsessecersereeenee 10) ® © © © 


60. How important is it to you to be a good reader? 
(1=not at all important, 3=somewhat important, 5=very important) 10) ® © 1) © 


Please tell us about the READING YOU DO AT HOME, using the following scale: 


1. None 

2. Less than an hour 

3. One up to three hours 

4. Three up to five hours 

5. More than five hours 
61. How much time have you spent reading a book for schoo! at......... a Oo © © ® ® 
home this week? 
62. How much time have you spent reading a book that was NOT............. ) 2) 0) ® © 
for school this week? 
63. How much time have you spent reading a newspaper this week’....... ® @ ©) @ © 
64. How much time have you spent reading a magazine this week?.......... 0) 0) © (Q) © 
65. How much time have you spent reading on the internet that................ 0) @) 6) (0) ® 
was for school work this week? 
66. How much time have you spent reading on the internet that...........00+ 0) ® © ® © 


was NOT for school work this week? 
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Please BUBBLE the number that shows how CONFIDENT you are in each item, using the 
following scale: 


1 2 3 d 5 
Not at all Confident Completely Confident 
67. | can figure out hard words when | am reading..........sesesessesesseeneseeeenens O) (0) © © ® 
68. | can sound out long words when i am reading.........ssssesecssseesesesreresesees ® ® © ® © 
69. | can recognize words easily when | am reading..........eseseseeseeneenenees O) © © ® © 
70. | can read and understand the textbooks for my ClaSS@S.........ccccece ® () © @® © 
71, | can read and understand the NEWSPaPel............ssseceseseseseseeeesetenenaeaes 0) ® © @® © 
72. | can read and understand magazine articles. ............csscsesssseeeeeseeeeeeee © @ © ® © 
73. | can read and understand information on the internet...........csssese oO 0) © ® © 


THANK YOU FOR COMPLETING THIS SURVEY! 
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Teacher First Name: 

Teacher Last Name: 

School: 

Date (including day of the week): 
Time: 

Observer: 


Notes: 


Teacher Observation Protocol 
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Model Component: Motivation and Quality Description: 
Engagement 


Area of Interest: 


Developing 


Adequate 


Exemplary 


[>| Teacher makes connections 
between learning activities and real 
world issues 


LC] Teacher varies instructional format, 
i.e. group work, lecture, and partner 
work. 


L] Student prior knowledge, interest 
and background used in determining 
content, 


El Technology is used to facilitate 
learning. 


ei Teacher provides students with 
private feedback. 


LJ Teacher facilitates and engages 
students in the problem solving 
process. 


| Students have autonomy for 
learning with opportunities to make 
meaningrul choices. 


Students do not appear engaged throughout most of 
the observation. 


Learning activities relate only minimally to real world 
issues. 


Issues are not necessarily relevant to students’ 
interest. Some attempts are made to connect 
content and activities to students’ prior knowledge or 
background, but connections are weak or do not seem 
to support learning 


Some attempts are made to vary instructional format, 
but variations (i.e. student collaboration) are not well 
structured, well+nanaged, or students do not 
participate equally 


Feedback is given, but it isnot connected to student 
goals or does not seem to support student self 
regulation 


Students make minimal choices or choices do not 
seem to support student engagement or learning 


Students appear engaged through some of 
the class period, but seem disengaged during 
other portions of the class. 


Teacher makes some connections to real 
world issues, and these connections support 
learning. 


Teacher makes some connections with 
instruction to students’ lives, interests, 
cultures, experiences and backgrounds, and 
these connections support student learning. 


Instructional format varies with regular 
transitions and is generally well-structured 
and managed; most students participate. 


Student set goals, and teacher provides 
feedback to students that support student 
self+egulation. 


Students make choices about either whatis 
read, learned or how they demonstrate their 
learning. 


All students appear highly engaged in learning during 
the full observation period. 


Real world issues are central to the student learning. 


Students’ interests, background, experiences, cultures 
and/or interests are the focus of the instruction and 
guide teacher and students decisions about learning 
and instruction. Instruction is highly relevant to 
students’ lives, backgrounds, cultures, and interests. 
Instruction affirms students’ cultural identity. 


Varying instructional formats are well-structured, well- 
managed, and students engage purposefully and 
equally 


Students set goals for their learning, receive feedback 
from the teacher that supports student self-regulation, 
and students exhibit self+egulation 


Students make choices about both what is learned 
(read and written), and how they demonstrate 
learning 
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Model Component: Strategic Processing 
Area of Interest: 


[__]Teacher explains cognitive strategies for 
comprehension. 


fre Teacher models using cognitive strategies 
for comprehension. 


| Teacher encourages and provides 
opportunities for students to practice using 
cognitive strategies for comprehension. 


[__] Teacher asks students to reflect on reading 
comprehension and strategy use. 


foundational reading skills. (i.e. fluency, 
decoding, word recognition, etc. } 


Notes (document specific strategies used): 


El Teacher provides explicit instruction in 


Description: 


Teacher mentions comprehension strategies, 
but does so briefly. Strategies are covered 
superficially. Teacher doesnot check for 
student understanding of comprehension 
strategies. 


Teacher does not effectively model reading 
strategies (i.e. modeling is confusing or 
unclear). 


Teacher suggests students use strategies but 
does not monitor student use. 


Students are asked to reflect on strategies, but 
how they should do that is unclear, and teacher 
does not check for this reflection. 


Strategy instruction is isolated (i.e. not 
connected to meaningful text; links are not 
made to other strategies or processes). 


Teacher explicitly explains reading 
comprehension strategies, 


Teacher models reading strategies. 


Teacher encourages and monitors use 
of reading strategies and checks that 
students are using them. 


Students are asked to reflect on 
reading strategy use. Students are 
clear about how to reflect on their 
strategy use. Teacher checks for this 
reflection. 


Teacher is beginning to integrate 
strategies (discusses more than one 
and how they work together). Teacher 
beginsto focus on application of 
strategies. 


Comprehension strategies are explicitly taught, and 
differentiated according to student need, Teacher 
frequently checks for student understanding. 


Teacher models reading, varying according to student 
need. 


Teacher encourages use of reading and writing 
strategies according to student need. Students use 
different comprehension strategies according to their 
need and the requirements of the text. Students work 
with different texts and different strategies according to 
where they are developmentally. 


Students reflect in their reading and writing on the 
strategies they use, and the reflection drives the 
subsequent instructional decision-making for the 
teacher. Teacher encourages students to use their own 
reflections to guide their reading and writing across 
content areas. 


Teacher encourages and supports students in integrating 
and applying multiple strategies across contexts. 
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Model Component nawucionalStateges  [Oesaipliw ——~—SCSCSCSCSC~C~“S“S*C“SC“‘“*“*S*S*S*“‘“‘“<;SSW 
Devcernkar Adequate Exemplary 


Area |Areaofinterest Interest: 


eal Teacher explicitly incorporates higher level 
questions. 


[J Teacher uses instructional tools to support 
student comprehension or vocabulary, i.e. 
anticipation guides, word storm. 


O Teacher encourages students to process 
vocabulary through visual, auditory, 
physical and/or emotional experiences, i.e. 
students workin groups to discuss new 
words or play games related to vocabulary. 


LD students are provided the opportunity to 
use their own words or nondinguistic 
representations to define new words. 


LD Teacher describes, explains and provides 
an example of new, key terms. 


[_] Teacher explicitly teaches and encourages 
students to use summarizing. 


[J Teacher explicitly teaches identifying 
similarities and differences. 
Notes (Be sure to list the names of specific tools 
that are used. )}: 


Instructional tools for comprehension or 
vocabulary are used, but they are not well- 
implemented (not clear to students how to use 
them) 


Vocabulary instruction is not well structured or 
does not seem to promote students’ vocabulary 
learning: instruction is not clear or does not 
seem to build students’ conceptual knowledge 


Explanation of strategies isnot clear. Students 
are not asked to apply them to other subjects or 
contexts. 


Students are exposed to tools but do not make 
connections to their use in other situations. 


The use of instructional tools supports 
students’ understanding of vocabulary , 
text, text or content to some extent; 
Tools are clearly explained 


Instruction builds students’ conceptual 
understandings and some transfer of 
learning to other contexts and subject 
areasis evident. 


Explanations of instructional strategies 
are clear and there is some evidence 
that students are learning to use them 
in other subjects or contexts 


Students have learned some 
instructional tools and teacher clearly 
communicates situations to use tools 


Instructional tools effectively used to support students’ 
learning. The teacher clearly explains how the tool is to 
be used, and instruction in the tool supports students’ 
independent comprehension and vocabulary 
development 


Vocabulary instruction supports students’ conceptual 
understandings; Instruction focuses on transfer of 
learning to other contexts and subject areas and 
students are encouraged to reflect on their use of new 
vocabulary or strategies in other contexts 


Explanations are clear, and scaffoldingis provided to 
support students’ growing independence 


Students have a large repertoire of tools from which to 
select. Students are able to appropriately select tools, 
and use them in other contexts or subjects. 
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Discussions are not well managed; | Discussions are well structured and Discussions are well structured and managed. The teacher promotes 
Some students do not participate in | well managed; There is some equitable social relationships, and students participate equally in 
discussions or are marginalized in participation from al students in discussions; discussion supports student learning of content. 
discussion groups; discussions do discussions; Discussions support 

not support student learning of students’ learning of content to some 

content. extent 


] Teacher uses pre-text based discussion 
strategies, i.e. teaching discussion skills 


CL] Teachers uses during text-based discussion 
strategies, i.e., question formation 


CL] Teacher uses post text-based discussion Writing to learn activities are used, | Writing to learn activities support The teacher focuses on students’ use of writing to learn strategies 
strategies, i.e. Student reflection but they are not clearly students’ understanding of texts or independently in different contexts and classes. 

implemented or do not seem to content 

support students’ understanding of 


Cc] Teacher includes writing to learn activities 
texts or content 


(i.e. any KCLM teaching tool that requires 
writing) 


L] Teacher explicitly teaches writing strategies. 


(i.e. pre-writing, outline, revising, etc.) Instruction of writing strategiesis Explicit instruction of writing includes | The teacher supports students’ independent use of writing strategies 
attempted, but strategies are not elements of modeling, explanation, and asks students to reflect on their writing strategies. 
[] Teacher explicitly teaches foundational clearly taught guided practice 
writing skills. (i.e. mechanics, DOL, grammar, 
etc.) 
Notes: 


Teacher does not discuss with Teacher discusses with students Teacher engages the students in modeling and engaging in effective 
students about “how” to effectively | about “how” to engagein discussion. | classroom discussionin which all students are active participants. 
engage in discussion 


Cantrell , Carter, & Rintamaa (2012) 


